The Power of Confidence Intervals

We connect the power of Confidence Intervals in different Frequentist methods to their reliability. We show that in the case of a bounded parameter a biased method which near the boundary has large power in testing the parameter against larger alternatives and small power in testing the parameter against smaller alternatives is desirable. Considering the recently proposed methods with correct coverage, we show that the Maximum Likelihood Estimator method [1, 2] has optimal bias. It is well known that the most important property of Frequentist Confidence Intervals is coverage: a Confidence Interval belong to a set of intervals that cover the true value of the measured quantity with Frequentist probability . Neyman’s method obtains Confidence Intervals with correct coverage through the construction for each possible value of of an acceptance interval with probability for an estimator of . The union of all acceptance intervals in the – plane is called the Confidence Belt. The Confidence Interval for resulting from a measurement of the estimator is the set of all values of whose acceptance interval for include . Coverage is not the only property of Confidence Intervals, because many methods for the construction of a Confidence Belt with exact coverage are available (see Refs. [3, 4, 5, 1, 2]). These methods differ by power [6], a quantity which is obtained considering the construction of acceptance intervals as hypothesis testing. Coverage and power are connected, respectively, with the so-called Type I and Type II errors in testing a simple statistical hypothesis !#" against a simple alternative hypothesis !%$ (see Ref. [3], section 20.9): Type I error: Reject the null hypothesis !#" when it is true. The probability of a Type I error is called size of the test and it is usually denoted by . Type II error: Accept the null hypothesis !#" when the alternative hypothesis !&$ is true. The probability of a Type II error is usually denoted by ' . The power of a test is the probability (*)+ ,' to reject !#" if !%$ is true. A test is Most Powerful if its power is the largest one among all possible tests. This is clearly the best choice. Unfortunately, the power associated with a confidence belt is not easy to evaluate, because for each possible value -" of considered as a null hypothesis there is no simple alternative hypothesis that allows to calculate the probability ' of a Type II error. Instead, we have the alternative hypothesis !%$ : $/. )0 -" , which is composite. For each value of $/. )1 -" one can calculate the probability ' 2435 6 $7 of a Type II error associated with a given acceptance interval corresponding to " . A method that gives an acceptance region for -" which has the largest possible power ( 283 6 $9 :); < =' 283 6 $ is Most Powerful with respect to the alternative $ . Clearly, it would be desirable to find a Uniformly Most Powerful test, i.e. a test that gives an acceptance region for -" which has the largest possible power ( 243> 6 $ for any value of $ . Unfortunately, the Neyman-Pearson lemma implies that in general a Uniformly Most Powerful test does not exist if the alternative hypothesis is two-sided, i.e. both $@? " and $ A " are possible, and the derivative of the Likelihood with respect to is continuous in -" (see Ref. [3], section 20.18). Nevertheless, it is possible to find a Uniformly Most Powerful test if the class of tests is restricted